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Abstract. Migration policies in distributed evolutionary algorithms has 
not been an active research area until recently. However, in the same 
way as operators have an impact on performance, the choice of migrants 
is due to have an impact too. In this paper we propose a new policy 
(named multikulti) for choosing the individuals that are going to be sent 
to other nodes, based on multiculturality: the individual sent should be as 
different as possible to the receiving population (represented in several 
possible ways). We have checked this policy on different discrete opti- 
mization problems, and found that, in average or in median, this policy 
outperforms classical ones like sending the best or a random individual. 



1 Introduction and state of the art 

Evolutionary algorithms (EAs) make individuals in the population evolve in 
parallel, which suggests that the exploitation of parallelism can be quite natu- 
ral in these algorithms. This has led to many efforts to parallelize them with 
the intention of reducing the execution time and improving the quality of the 
solutions. There have been several attempts to classify the numerous works de- 
voted to parallel EAs |1I2| . with island or coarse grained models being one of 
the most popular approach of parallelization of EAs. This approach is usually 
implemented on distributed systems, since it does not require a high cost in com- 
munications: nodes only exchange a few individuals after several generations. 
The population is divided in subpopulations, which usually evolve isolatedly ex- 
cept for migrations, that is, the exchange of some individuals after a number of 
generations. The behavior of the island model differs from the sequential EA, 
since the composition, and thus the dynamics, of every subpopulation is differ- 
ent: since subpopulations are smaller than the whole population, the parallel EA 
will converge faster. Furthermore, migrations among subpopulations usually im- 
prove the quality of the sequential solutions [3] , which makes the parallel model 
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interesting even for sequential executions. Not only does this apply to genetic 
algorithms; similar results have been found for Genetic Programming: Tomassini 
et al. [4] nave analyzed diversity in multipopulation genetic programming (GP) 
finding a correlation between diversity and the better convergence properties of 
distributed GP. 

These results have inspired this work. Diversity in the subpopulation is so 
important that it leads to improvement in quality and efficiency at the same 
time. Accordingly, we have looked for the way of enhancing the diversity induced 
by the usual migration policies, by using the notion of multiculturality: migrants 
should be chosen on the basis of genotypic difference to the receiving population; 
we have called this new migrant-selection policy multikulti 1 . 

Let us then look at how these diversity enhancement could be realized through 
migration policies, which include several issues: 

— the number of individuals undergoing migration, 

— the frequency of migration, i.e. the number of generations or evaluations 
between migrations, 

— the policy for selecting migrants, 

— the migration replacement policy, 

— the topology of the communication among subpopulations, 

— the synchronous or asynchronous nature of the communications. 

These issues have been investigated in different papers: Alba et al. [3j compare 
synchronous and asynchronous migration policies, Herrera et al. [6] studied some 
of the aforementioned issues in a hierarchical configuration of subpopulations, 
and Cantii-Paz |7l8j . Alba and Troya [9], and Noda et al. [10] have analyzed 
different migration policies. Several results presented in these mentioned works 
indicate that diversity is a fundamental key in the success of the island model. 
For example, works comparing synchronous versus asynchronous models [5] , have 
found that the asynchronous algorithms outperformed the synchronous ones in 
all the experiments. Cantu-Paz [718] has studied the four possible combinations 
of random and fitness-based emigration and replacement of existing individuals. 
He found that the migration policy that causes the greatest reduction in work 
(takeover time 2 ) is to choose both the migrants and the replacements according 
to their fitness, because this policy increases the selection pressure and may cause 
the algorithm to converge significantly faster. However, if convergence is too fast 
it can lead to algorithm failure, as Cantu-Paz [8] states referring to parallel EAs: 

Rapid convergence is desirable, but an excessively fast convergence 
may cause the EA to converge prematurely to a suboptimal solution. 

So, other policies must also be considered. In fact, Alba and Troya [9] found 
that migration of a random string prevents the "conquest" effect in the target 

1 Multikulti, as denned by the wikipedia entry 
(http://en.wikipedia.org/wiki/Multikulti), is an slogan for a multicultural 
approach to public policy 

2 Number of generations required to converge to the best individual from the initial 
population, by applying selection only 
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island for small or medium sized sub-populations. Noda et al. [10] have proposed 
choosing which individuals to migrate and/or replace adaptively depending on 
some knowledge-oriented rules. To do this, each agent receives information about 
the fitness function from its peers. The tested adaptive policies have proved useful 
providing best solutions than the sequential execution. 

In spite of the results shown above, there is still a number of issues that have 
not been investigated yet. In this work we also focus on the policy for migrants 
selection. Previous works dealing with this aspect have studied the use of any of 
the selection operators usually applied in evolutionary algorithms: proportional 
selection, tournament, random, etc. Only the work by Noda et al. [10] considers, 
among other policies, one in which the individuals sent are chosen to be quite 
different from others previously sent. 

The aim of this work is to exploit differences in the various subpopulation. To 
do this, we focus on the selection of the individuals to be sent to other subpopula- 
tion. Our thesis is that migrating individuals different enough to the destination 
subpopulation instead of the best individuals can result in a better performance 
through the diversity enhancement it produces. Consider the Figure [T] with two 
subpopulations. The black points represent the distribution of the population 
along the function to optimize. Individual a in subpopulation PI has the high- 
est fitness, and thus it would be sent to subpopulation P2 following the most 
common migration policies. We propose to send individual b, whose genotype 
is quite different from those of subpopulation P2. In the example it would lead 
to exploration of a new area of the search space where the global optimum is 
placed. In order to achieve this, the process corresponding to subpopulation PI 
needs to receive information on the composition of the individuals in subpopu- 
lation P2. We have considered different ways of providing this information in a 
concise maimer. One of them is taken the best individual of subpopulation P2 
as representative. The other one is using a kind of average genotype, the con- 
sensus sequence described later, as representative of subpopulation P2. Another 
important issue to investigate if the trade-off between promoting diversity and 
favoring the best individuals. The risk of sending the most different individual 
as migrant is that if its fitness value is low compared to those of the destination 
subpopulation, the migrant would probably disappear immediately. Therefore, 
another question tested in the experiments has been if the most different indi- 
vidual is fit enough to survive when migrating, or if it is best to select the most 
different from an elite. 

The rest of the paper proceeds as follows: section 2 describes the model de- 
tails; section 3 is devoted to describe the evolutionary algorithm and its imple- 
mentation; section 4 presents and discusses the experimental results, and section 
5 draws the main conclusions of this work. 

2 Model Description 

We have considered a ring topology (Figure [2]), in which each node can send one 
or more individuals to the next node in the ring. To perform the choice of the 
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PI P2 




Fig. 1. Illustration of the multikulti migrant selection policy: Individual a in 
subpopulation PI has a higher fitness than individual b. However b is more 
different to the best individual in subpopulation P2. Accordingly, b is selected 
to be sent in the migration. 




Fig. 2. Scheme of the multikulti algorithms. 



migrants, the node P, receives from node Pj+i information about the genotype 
of its subpopulation. We have considered two different ways of representing this 
information in a concise manner: 

1. With the best individual of the subpopulation. After a number of generations 
without exchanging individuals we can expect that each subpopulation is 
close enough to converge for the best individual being a fair representation 
of the whole population. 

2. With the consensus sequence of the population. This is a concept taken from 
biology where it is defined as the sequence that reflects the most common 
choice of base or amino acid at each position of a genome. Areas of par- 
ticularly good agreement often represent conserved functional domains. In 
our case it is composed of the most frequent alelo for each position of the 
genotype. 

Once the node Pj has got this information, it sends to node Pj+i an individ- 
ual different enough from the subpopulation P i+ i representative. Here we have 
considered two approaches: 

— Base: Selecting the most different from the subpopulation Pi. 

— Elite: Selecting the most different among the best half of subpopulation p. 



5 



3 Implementation 

Chromosomes pj] of our GA are fixed- length binary strings. The selection mech- 
anism to choose individuals for the new population uses a steady state algorithm, 
with two-point crossover operator and single-bit-flip mutation. The rest of the 
parameters are shown in tables []Ja) and [ljb) . 







Parameter 


Value 


Parameter 


Value 


Chromosome length 


120 


Population 


32 


Population 


256 


Selection rate 


60% 


Selection rate 


20% 


Generations to migration 


20 


Generations to migration 


20 


Mutation priority 


2 


Mutation priority 


2 


2-point crossover priority 


3 


2-point crossover priority 


3 






Max number of evaluations 


200000 


(a) 




(b) 





Table 1. Evolutionary algorithm parameters used in the P-Peaks (a) and in the 
MMDP(b) experiments. 



P-Peaks and the massively multimodal deceptive problem (MMDP), two of 
the three discrete optimization problems presented by Giacobini et al. in [12] 
have been selected for testing. These problems, while being both multimodal, 
represent different degrees of difficulty for parallel evolutionary optimization. 
They will be described below. 

These two problems have been implemented and integrated in the Algo- 
rithm: : Evolutionary library, which is freely available under the GPL license 
from http : //opeal . cvs . sourcef orge . net/opeal/Algorithm- Evolutionary/ 
In order to simulate a parallel algorithm, the cooperative multitasking Perl mod- 
ule POE has been used; each node is represented by a POE session. The rest 
of the evolutionary algorithm has been implemented using the same Algo- 
rithm : : Evolutionary Perl module j!3j . The program, along with the parameter 
sets used, is also available under an open source license from the same site. 

In this simulated parallel scenario, each node runs a rank-based substitution, 
steady state algorithm. At the end of a preset number of generations, each node 
sends a single individual to the other node according to the policy being tested 

3.1 Problems tested 

Two functions have been used for testing: P-Peaks and MMDP, two of the three 
discrete optimization problems presented in 12 : The massively multimodal de- 
ceptive problem (MMDP) and the problem generator P-Peaks. These problems, 
while being both multimodal, represent different degrees of difficulty for parallel 
evolutionary optimization. They will be described next. 
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The MM DP [14] is a deceptive problem compose of k subproblems of 6 bits 
each one (si). Depending of the number of ones (unitation) s, takes the values 
depicted next: 

fitness Si (0) = 1.0, fitness Si (l) = 0.0, fitness Sz {2) = 0.360384, 
fitness Si (3) = 0.640576 fitness = 0.360384, fitness Sz (5) = 0.0, 
fitness Si (6) = 1.0 

The fitness value is defined as the summatory of the Si subproblems with an 
optimum of k (equation!]}. The number of local optima is quite large (22 fc ), while 
there are only 2 k global solutions. In this paper, we consider a single instance 
with k = 20. 

k 

fMMDp(s) = ^2 fitness Sz (1) 

i=l 

The P- Peaks problems is a multimodal problem generator proposed by De Jong 
in p~5] ; and is created by generating P random N — bit strings where the fitness 
value of a string x is the number of bits that x has in common with the nearest 
peak divided by N. In the experiments made in this paper we will consider 
P = 100; the optimum fitness is 1.0. 

fp-Peaks(x) = — max {N — Hamming Distance(x, Peaki)} (2) 

iv l<i<p 

We consider an instance of P = 100 and 64 bits where the optimum fitness is 
1.0 (Equation 02). 

These two problems have been also implemented and integrated in the Al- 
gorithm: :Evolutionary library. 

4 Experimental results 

First, we tested several parameter configurations for the P-Peaks problem. In 
general, when diversity conditions are not too harsh, the performance difference 
between different migration policies is not too high. Eventually, when the going 
gets tough, differences such as those shown in Figure[3]do appear. In that graph, 
taken for a 8-node, population = 32 experiment, results are quite different de- 
pending on the migration policy. For starters, sending the best individual yields 
the worst results. If we consider the median, the multikulti is similar to the ran- 
dom policy, but its behavior is better if we consider the average and the worst 
case, as shown in table [5J 

The MM DP was also tested with a similar setup; results are shown in figure 
[4] where three new versions of the algorithm, named multikulti-elite, consensus- 
multikulti and multikulti-elite-consensus, have been tested. Multikulti-elite chooses 
the individual most different to the receiving population best individual, but 
only among the 50% best. In this case, not surprisingly, this strategy beats the 
multikulti by far as well as the random strategy, but more closely. This is proba- 
bly due to the nature of the MM DP: it is a deceptive problem, where increasing 
diversity might not have the desired result, since competing conventions for the 
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Policies comparison, P-Peaks problem 
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Fig. 3. Boxplot (with logarithmic y axis) of the number of evaluations needed 
to find the solution in the P-Peaks problem, best represents the behavior of the 
experiment when the best individual in the population is sent, random with a 
random individual, and finally mk, in the middle, stands for multikulti, the algo- 
rithm we are testing in this paper which sends the individual in the population 
most different to the best in the receiving population. 



same 6-bits portion can lead in each population. This why, instead of sending the 
most different as the multikulti policy does, doing it with one that is different 
enough, but at the same time, fit enough, produces the best results. 

The second strategy tested, consensus-multikulti, achieves an intermediate 
performance among the two, beating the multikulti, but achieving worse results 
than multikulti-elite. This strategy sends the individual that is most different to 
the consensus string, that is, the string whose every bit represents the majority 
value for that position among the population. The result is probably due to the 
same reason: a value that is too different represents a high disturbance, and thus 
is bad for diversity. In fact, the third strategy, multikulti-elite-consensus, which 
sends the individual most different to the consensus string which is among the 
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Policy 



Median 



Average 



Best 

Random 

Multikulti 



25820 
1545 
1544 



25310 
6410 
1252 



Table 2. Statistics for number of evaluations of different migration policies for 
the P-Peaks problem. 



50% best, achieves results similar to Multikulti-elite, although, in general, a bit 
better. 

In order to investigate what is going on, we have measured the entropy for the 
MM DP. The results are shown in Figure EJ which shows the different evolution 
paths of phenotypic entropy (computed using the Shannon formula) with the 
multikulti-elite migration policy (left) and the best migration policy (center). 
The behavior es quite different. The multikulti policy, not only keeps the entropy 
high, but considerably increases it in some populations during evolution. The 
policy of migrating the best provides quite much lower levels of entropy; with a 
decreasing trend that never changes, leading to a collapse of entropy from cycle 
12. This proves the utility of the multikulti policy to maintain diversity, and 
supports the result that the improvement in the number of evaluations is due 
precisely to this diversity-enhancing effect brough by the multikulti policies. 

5 Conclusions 

This paper has explored new alternatives to promote diversity in an island model. 
This is achieved by selecting as migrant individuals with a genotype different 
enough to the destination population. Because there is a trade-off between pro- 
moting diversity and favoring the best individuals, we have performed experi- 
ments to find out the degree of difference which produces the best results. These 
experiments have shown that results only improve substantially if the migrant 
is chosen from the elite; that is, those with an above-average fitness, which in- 
dicates that diversity only improves the results if the incoming migrant has a 
minimum level of quality. We have compared two different ways of characterizing 
the destination population: the best individual and the consensus sequence. Re- 
sults have shown that both of them represent appropriately the population, with 
the consensus sequence performing only slightly better. Studying the phenotypic 
entropy of the population we have found that our method effectively improves 
entropy by avoiding entropy to fall too fast and also creating an entropy differ- 
ential among populations, and thus diversity. 

In the future we intend to develop a parallel implementation of the system, 
which will allow us to measure also execution times. We are also working on al- 
ternative mechanisms to characterize the destination population, and thus select 
the more appropriate migrants. We will also test results obtained by changing 
other algorithm parameter such as number of migrants or the number of nodes. 
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Policies comparison, MMDP problem 
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Fig. 4. Boxplot (with logarithmic y axis) of the number of evaluations needed to 
find the solution for MMDP. Three different versions of the multikulti migration 
policy have been tested here: mk, in which the most different individual is sent; 
mk-cons, that sends the most different to the target's consensus string, multikulti- 
elite (mke) which chooses the most different among the 50% most fit, and mke- 
cons which sends the most different to the target's consensus string among the 
50% most fit. The rightmost result corresponds to random migration. 
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Fig. 5. Entropy (computed using the Shannon formula H{P) = 
-J2 g epP(f(g)) l °9bP(f(g)), where g is a member of the population, f(g) 
its fitness, and p(f(g)) the frequency of that fitness across the whole popula- 
tion) in a typical run of the MM DP problem, with the multikulti-elite migration 
policy (left) and the best migration policy (center). Every line corresponds 
to a different population, of the eight running in parallel. The figure on the 
right compares average values, with the dashed line corresponding to the 
multikulti-elite experiment and the other to the experiment that sends the best 
individual. 



